1. Field of the Invention
This invention relates to a system and method for imaging a scene with plural sensors sensitive to different scene characteristics, determining the best features received from each sensor and then fusing or merging imagery of the best features from the multiple sensors to provide an image having improved information content.
2. BRIEF DESCRIPTION OF THE PRIOR ART
Image sensors employed in present day military and scientific environments attempt to extract as much information about a scene under observation as possible. In order to perform this function, it is necessary to interrogate the scene being observed with as many different types of sensors as is feasible. Visible, infra-red and image intensified sensors represent three of the most common passive imaging sensors utilized in the military environment. Each sensor detects different information about the imaged scene.
It is possible to present the operator with multiple simultaneous displays, one from each of the sensors, or allow the operator to switch between or among the sensor outputs. However, displaying all of the information content from each of the sensors on a single composite display represents a far superior approach from an operator workload standpoint.
Known existing approaches to the above noted problems either do not lend themselves to real-time implementations or result in critical information loss or distortion. Known approaches are:
1. Adding or averaging the multiple images. This approach has the potential for critical information loss. As an example, if two images contain the same object but of equal magnitude and opposite polarity, they will cancel one another out in the resultant image.
2. Level based keying wherein the level of one image is used as the criterion for switching to the other image. This approach results in ragged edge artifacts when the other image is switched in. It also does not insure that the switched image will have any better information content than the prior image.
3. Transform based approaches which technical literature describes as several transform based techniques such as the Hotelling Transform approach. These approaches have been primarily developed for merging satellite photographs from different spectral sensors. These techniques do not lend themselves to real-time implementations.
4. ROLP (Ratio of Low-Pass) pyramid which is based upon successive lowpassing and decimation. Decimation is a common digital signal processing technique for downsampling or sample rate reduction. For example, if a signal is decimated by 4, every fourth sample is retained and the rest are discarded. It again does not lend itself to reasonable real-time hardware implementations.